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FOREWORD 


A  joint-service  coordinated  effort  is  in  progress  to  develop  a  computerized  adaptive 
testing  (CAT)  system  and  to  evaluate  its  potential  for  use  in  the  military  entrance 
processing  stations  as  a  replacement  for  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB)  printed  tests.  The  Department  of  the  Navy  (Headquarters,  U.S.  Marine  Corps) 
has  been  designated  as  lead  service  for  CAT  system  development;  and  the  Navy  Personnel 
Research  and  Development  Center,  as  lead  laboratory. 

This  report  is  the  sixth  in  a  series  being  issued  under  the  CAT  project.  Previous 
reports  described  CAT  system  functional  requirements  and  schedules,  preliminary  design 
considerations,  the  influence  of  fallible  item  parameters  on  adaptive  testing,  the  relation¬ 
ship  between  corresponding  ASVAB  and  CAT  sub  tests,  and  a  theoretical  foundation  for 
adaptive  administration  of  aptitude  tests  (NPRDC  Tech.  Note  82-22  and  Tech.  Reps.  82- 
52,  83-15,  83-27,  and  83-32). 

This  report  describes  the  development  of  the  Computerized  Adaptive  Screening  Test 
(CAST),  a  screening  test  to  be  used  at  recruiting  stations  to  predict  ASVAB  performance. 
CAST  was  developed  in  support  of  Advanced  Computerized  Adaptive  Testing  (Z1385- 
PN.01)  and  was  described  previously  in  NPRDC  Spec.  Rep.  83-36.  Results  are  intended 
for  use  by  the  research  community  and,  potentially,  by  all  of  the  armed  services. 


3.  W.  RENARD 
Captain,  U.S.  Navy 
Commanding  Officer 


3.  W.  TWEEDDALE 
Technical  Director 
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SUMMARY 


Problem 


Armed  services  recruiting  faces  serious  challenges  in  the  future,  due  to  a  shrinking 
pool  of  military  eligibles.  Added  to  vastly  increased  expenditures  in  recruiting  caused  by 
the  move  to  an  all-volunteer  force  (AVF)  will  be  a  fierce  and  costly  competition  for 
available  personnel  among  colleges  and  the  several  armed  forces,  which  will  increase  the 
difficulty  of  recruiting.  The  best  available  candidates  for  enlistment  must  be  located, 
enlisted,  and  optimally  assigned.  Fiscal  and  personnel  resources  cannot  be  wasted;  thus, 
recruiter  tasks  that  detract  from  the  primary  mission  of  locating  prospects  and  "selling 
the  service"  must  be  reduced. 

Objective 

The  objective  of  the  effort  reported  here  was  to  design  and  develop  the  Computer¬ 
ized  Adaptive  Screening  Test  (CAST)  that  could:  (1)  operate  on  a  stand-alone  micro¬ 
computer  system  in  recruiting  stations,  (2)  reduce  the  recruiters'  administrative  burden, 
and  (3)  predict  applicants'  scores  on  the  Armed  Forces  Qualification  Test  (AFQT)  more 
efficiently  than  the  Enlistment  Screening  Test  (EST),  a  conventionally-administered, 
paper-and-pencil  test  used  at  present. 

CAST  Design  and  Development 

1.  Item  banks  were  developed  for  the  three  CAST  subtests  (Word  Knowledge, 
Arithmetic  Reasoning,  and  Paragraph  Comprehension).  An  ability  estimation  procedure 
for  scoring  and  selecting  test  items  was  chosen,  and  a  stopping  rule  for  ending  the  test 
was  determined. 

2.  The  interactive  dialogues  were  written  for  presentation  on  the  video  display 
terminal.  Computer  programs  and  software  documentation  were  written  for  administer¬ 
ing,  scoring,  and  interpreting  CAST  on  the  microcomputer  equipment  at  NAVPERSRAND- 
CEN. 


3.  The  CAST  subtests  were  pilot-tested  and  statistical  analyses  were  performed  on 
the  resulting  data. 

Results 


In  predicting  AFQT  scores,  CAST  was  essentially  as  effective  as  the  conventionally- 
administered,  paper-and-pencil  EST,  while  using  fewer  items. 

Conclusions 

CAST  should  prove  superior  to  the  EST  in  terms  of  administration  and  management 
and  has  been  shown  to  be  comparable  in  predictive  power.  It  will  relieve  recruiters  of 
clerical  burdens  associated  with  applicant  screening,  making  additional  time  available  for 
effective  recruiting  duties.  Suitable  for  use  with  computer-naive  personnel,  CAST  will 
improve  test  security,  reduce  compromise,  and  reduce  one  source  of  recruiter  mal¬ 
practice.  It  will  enhance  the  image  of  military  recruiting  and  place  the  implementing 
service  in  a  forward  position  regarding  addition  of  other  screening  instruments,  test 
revision,  and  so  on. 
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Recommendations 


It  is  recommended  that  the  recruiting  command  of  the  using  armed  service: 

1.  Administer  CAST  to  service  applicant  populations.  Analyses  could  be  conducted 
on  the  data  obtained  to  establish  a  cutting  score  for  use  in  determining  whether  to  send  an 
applicant  to  a  military  entrance  processing  station  or  mobile  examining  test  site  for 
ASVAB  testing. 

2.  Investigate  the  suitability  of  the  CAST  interactive  dialogue,  system-user  inter¬ 
face,  and  its  effect  on  recruiting  operations. 

3.  Refine  CAST,  as  appropriate,  to  achieve  recruiter  and  applicant  acceptance. 
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INTRODUCTION 


Background  and  Problem 

Armed  services  recruiting  faces  serious  challenges  in  the  future,  due  to  a  shrinking 
pool  of  military  eligibles  (Congressional  Budget  Office,  1980).  The  all-volunteer  force 
(AVF)  concept  has  led  to  vastly  increased  expenditures  in  recruiting  (Office  of  Naval 
Research,  1979).  The  results  will  be  a  fierce  and  costly  competition  for  available 
personnel  among  colleges  and  the  several  armed  forces,  which  will  increase  the  difficulty 
of  recruiting  (Joint  Chiefs  of  Staff,  1982).  The  best  available  candidates  for  enlistment 
must  be  located,  enlisted,  and  optimally  assigned.  Fiscal  and  personnel  resources  cannot 
be  wasted;  thus,  recruiter  tasks  that  detract  from  the  primary  mission  of  locating 
prospects  and  "selling  the  service"  must  be  reduced. 

The  screening  of  applicants  for  enlistment,  which  takes  place  near  the  end  of  the 
recruiting  process,  includes,  as  a  major  element,  the  administration  of  the  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB),  a  10-test  battery  given  to  all  armed  services 
applicants.  To  be  eligible  for  enlistment,  an  applicant  must  achieve  a  minimum  Armed 
Forces  Qualification  Test  (AFQT)  score,  which  is  a  linear  composite  of  the  scores 
obtained  on  four  ASVAB  tests.  The  ASVAB  is  given  to  a  large  number  of  high  school 
students  through  the  Department  of  Defense  High  School  Testing  Program  and  to  service 
applicants  (unless  they  have  previously  achieved  a  qualifying  AFQT  score)  at  a  military 
entrance  processing  station  (MEPS)  or  mobile  examining  test  (MET)  site. 

Transportation  from  a  recruiting  station  to  a  MEPS/MET  site  entails  costs  for  the 
transportation  and,  in  many  cases,  for  meals  and  lodging.  Costs  are  also  incurred  for 
personnel  time  at  both  the  recruiting  station  and  the  test  site.  If  applicants  are  sent  to 
the  MEPS/MET  site  and  subsequently  fail  the  ASVAB,  there  is  a  significant  waste  of 
money.  Conversely,  if  applicants  who  would  have  passed  are  erroneously  denied  ASVAB 
testing,  their  talents  are  lost  to  the  service,  social  costs  accrue  to  the  applicants,  and  (in 
a  tight  recruiting  market)  pressure  on  recruiters  to  obtain  a  replacement  applicant  is 
increased. 

Aptitude  screening  to  predict  the  AFQT  score  originated  because  of  the  need  for 
expenditure  control.  However,  since  results  are  used  early  in  the  recruiting  process  to 
decide  whether  an  applicant  should  be  sent  for  ASVAB  testing  or  rejected,  the  accuracy  of 
those  results  have  great  impact  on  both  the  armed  services  and  the  applicant. 
Accordingly,  aptitude  testing  assumes  a  critical  importance  (Maier  <5c  Fuchs,  1973). 

Currently,  all  armed  services  use  the  Enlistment  Screening  Test  (EST)  at  recruiting 
stations  to  predict  applicant  AFQT  scores.  EST  is  composed  of  3  subtests  totaling  48 
items,  with  a  total  time  limit  of  45  minutes.  It  was  developed  by  the  Air  Force  Human 
Resources  Laboratory  in  1976  (Jensen  &  Valentine,  1976)  and  revised  in  1981  (Mathews  & 
Ree,  1982). 

Pure  measurement  error  does  not  seem  to  be  a  problem  with  the  EST;  it  appears  that 
the  newer  test  forms  (81a  and  81b)  provide  adequate  measures  throughout  the  score  range 
where  most  selection  and  classification  decisions  are  made  and  it  correlates  .83  with  the 
AFQT  (Mathews  &  Ree,  1982).  However,  EST  does  share  in  the  problems  of  adverse 
psychological  effects.  It  is  a  conventionally-administered,  paper-and-pencil  type  of  test, 
which  has  been  shown  to  be  associated  with  high  levels  of  guessing,  frustration,  and 
boredom  in  subjects  (Vale  &  Weiss,  1975;  Weiss,  1974).  In  their  customary  delivery  mode, 
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conventional  paper-and-pencil  tests  actually  combine  the  worst  features  of  individually- 
administered  tests  (i.e.,  they  are  heavily  dependent  upon  variables  associated  with  the 
examiner  or  the  examiner-examinee  relationship)  and  group  tests  (i.e.,  item  arrangement, 
item  set,  answer  sheet  effects,  and  imprecision)  (Weiss  <5c  Betz,  1973).  EST  is  scored  on  a 
pass/fail  basis.  Since  EST  is  also  a  timed  test,  it  exerts  differential  pressure  on  the 
examinees  (i.e.,  results  are  based  partly  upon  individual  reaction  >to  time  constraints 
(Weiss,  1974)). 

Major  concern  focuses  on  administrative  error  and  clerical  burden.  EST  requires 
approximately  45  minutes  to  administer,  plus  the  time  required  to  score  and  interpret 
results  and  to  manage  the  test  supplies  (by  a  recruiter  already  investing  many  hours  in  the 
potential  enlistee).  At  present,  besides  storing,  filing,  retrieving,  and  ordering 
replacements,  the  recruiter  is  required  to  take  frequent  inventory,  make  numerous  checks 
and  corrections  for  unauthorized  markings  in  the  test  booklets,  and  safeguard  used  answer 
sheets.  EST  is  thus  highly  labor  intensive,  consuming  the  time  of  a  senior 
noncommissioned  officer  in  quasi-clerical  tasks. 

Since  only  two  forms  of  EST  are  in  use,  the  failing  subject  can  hope  to  pass  the  test 
eventually  by  repeated  testing  and  item  memorization.  If  more  contraindicators  were 
needed,  many  are  apparent:  (1)  the  initial  and  replacement  costs  and  short  life  of 
materials  associated  with  paper-and-pencil  tests,  (2)  the  poor  impression  created  by 
dilapidated  materials,  and  (3)  security,  custody,  and  control  difficulties.  In  short,  to 
reduce  the  recruiter's  task  load,  there  is  a  critical  need  for  a  predictive  instrument  that  is 
more  effective  and  a  screening  method  that  is  easier  to  administer. 

Computerized  adaptive  testing  (CAT)  (McBride,  1979)  combines  recent  developments 
in  latent  trait  theory  with  the  ever-increasing  power  and  efficiency  of  computers.  The 
result  is  employment  of  individualized  testing  without  the  loss  of  administration  ef¬ 
ficiencies  gained  through  group  testing,  combining  advances  in  computer  technology  with 
those  in  psychometrics,  and  using  computers  to  administer  tests  that  adapt  themselves  to 
individual  differences  in  ability  levels  during  their  administration  (Pine,  Church,  Gialluca, 
<Sc  Weiss,  1979).  The  difficulty  level  of  an  adaptive  test  is  dynamically  tailored  to  the 
ability  of  each  subject.  Also,  because  each  subject  receives  a  tailored  subset  of  items, 
the  chances  of  test  compromise  through  copying,  memorization,  or  pretest  coaching  are 
lessened. 

With  adaptive  testing,  shorter  tests  may  be  used  without  loss  of  reliability  or  validity 
(Betz  &  Weiss,  1975;  Weiss,  1974).  Adaptive  tests  are  untimed,  reducing  pressure  on  the 
subject  without  hindrance  to  the  proctor  (Weiss,  1976).  Adaptive  testing  reduces  boredom 
and  frustration  (Weiss,  1974),  guessing  (Betz  <5c  Weiss,  1975),  real  or  perceived  proctor- 
subject  bias  (Gorman,  1977),  and  culturally-specific  racial  bias  (Pine,  1977).  Adaptive 
testing  is  more  motivating  than  conventional  testing,  thereby  eliciting  "best  results"  from 
subjects  (Betz  &  Weiss,  1976)  and  more  accurately  reflecting  subject  competence  (Pine, 
1977). 

Computerized  testing  has  its  own  merits.  It  may  lessen  test  bias  through  item 
selection  and  increase  test  fairness  by  the  nature  of  the  test  itself  and  the  test's 
administration  modality  (Pine  et  al.,  1979).  The  computer  makes  it  possible  to  eliminate 
printed  test  materials  and  their  associated  logistical,  security,  and  administrative 
problems,  while  facilitating  item  replacement,  whole  test  construction,  and  the  capturing 
of  data  for  validation  purposes. 

After  each  CAT  question,  the  computer  can  use  the  response  information  to  update 
the  ability  estimate  and  then  use  the  new  estimate  of  ability  to  select  the  next  item. 
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With  each  successive  response,  the  ability  estimate  gains  reliability.  The  process  can 
continue  until  some  stopping  rule  is  satisfied  (e.g.,  a  fixed  number  of  questions 
administered  or  a  prespecified  level  of  reliability).  CAT  can  shorten  testing  time  without 
a  loss  of  test  effectiveness,  as  well  as  eliminate  scoring  and  recording  errors  due  to 
clerical  error  (Gorman,  1977). 

Objectives 


The  objectives  of  the  effort  reported  here  were  to  design  and  develop  a 
Computerized  Adaptive  Screening  Test  (CAST)  that  could:  (1)  operate  on  a  stand-alone 
microcomputer  system  in  recruiting  stations,  (2)  reduce  the  recruiters'  administrative 
burden,  and  (3)  predict  applicants'  scores  on  the  AFQT  at  least  as  accurately  as  the  paper- 
and-pencil  EST  used  at  present. 


CAST  DESIGN  AND  DEVELOPMENT 

Designing  and  developing  CAST  required  parallel  work  in  psychometrics  and  computer 
programming. 

Test  Construction 


CAST  was  envisioned  as  incorporating  three  subtests  that  would  correspond  to  three 
of  the  four  ASVAB  tests  used  to  calculate  the  AFQT  composite  score:  Word  Knowledge 
(WK),  Arithmetic  Reasoning  (AR),  and  Paragraph  Comprehension  (PC)  (Sands,  198 1). 1  Item 
banks  for  the  CAST  subtests  were  assembled  in  conjunction  with  the  University  of 
Minnesota  for  use  in  CAT  ASVAB  research  (Moreno,  Wetzel,  McBride,  &  Weiss,  1983)  and 
were  made  available  for  the  present  research  effort.  These  item  banks  included  78  WK 
items,  247  AR  items,  and  25  PC  items,  together  with  the  estimates  of  three  parameters 
(discrimination,  difficulty,  and  guessing)  for  each  item.2  A  Bayesian  ability  estimation 
procedure  described  by  Jensema  (1977)  was  chosen  for  scoring  and  determining  the 
selection  and  presentation  sequence  of  test  items.  The  stopping  rule  selected  was  the 
administration  of  a  fixed  number  of  items. 

Computer  System  Hardware  and  Software 

Computer  hardware  used  for  development  of  CAST  included: 

1.  Applied  Computer  Systems  (ACS)  microcomputer. 

2.  Perkin-Elmer  Data  Systems  1200  video  display  terminal  (VDT). 

CAST  computer  programs  were  written  to  provide  interactive,  user-friendly  software 
that  presumed  no  previous  computer  experience  on  the  part  of  either  recruiter  or 
applicant.  In  addition,  all  VDT  screen  text  displays  were  written,  reviewed,  and  edited  for 
readability  (reading  grade  level)  standards.  Figures  1  and  2  illustrate  a  sample  of  an 
instructional  display  and  a  demonstration  test  item.  To  facilitate  user  acceptance,  the 
time  between  item  response  and  presentation  of  the  next  item  generally  should  not  exceed 
3  seconds.  This  goal  was  accomplished. 


JThe  fourth  test  is  Numerical  Operations  (NO);  the  AFQT  formula  is  AR  +  WK  +  PC  + 
.5NO. 

2The  AR  test  item  bank  was  reduced  to  225  items  after  removal  of  superfluous  items. 
The  WK  and  PC  item  banks  remain  at  78  and  25  items  respectively. 
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You  will  now  be  given  the  Computerized  Adaptive  Screening  Test 
(CAST).  This  is  an  aptitude  test  designed  to  predict  how  well  you 
will  do  on  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB). 

In  order  to  qualify  for  enlistment,  you  must  get  a  passing  score  on 
the  ASVAB.  The  test  you  will  now  take  will  give  you  an  idea  of 
how  well  you  can  expect  to  do  on  the  ASVAB. 

***Press  GO  to  continue. 


Figure  1.  Sample  of  an  instructional  display. 


The  following  item  is  for  practice  only  and  will  not  be  counted  in 
scoring  your  test. 

Children  enjoy _ in  the  sandbox  at  the  park. 

1.  Understanding. 

2.  Finding. 

3.  Working. 

4.  Playing. 

Your  answer  is? 


Figure  2.  Sample  of  a  demonstration  test  item. 


Software  documentation  for  presentation  of  CAST  on  the  ACS  microcomputer  system 
was  completed.  Subsequently,  the  program  was  converted  to  operate  on  an  Apple  II-Plus 
microcomputer  with  two  disk  drives  and  a  VDT. 

Test  Administration 


In  each  CAST  subtest,  a  provisional  ability  estimate  is  made,  a  test  item  appropriate 
to  that  ability  level  is  presented,  and  the  ability  estimate  is  updated  based  on  the  response 
to  the  test  item.  The  computer  program  for  this  iterative  ability  estimation  process 
starts  by  associating  each  examinee  with  an  ability  level.  Selection  of  a  test  item  starts 
from  the  top  (highest  information  value  within  a  level)  and  searches  for  the  first  item  not 
yet  presented.  The  item  that  results  from  this  search  is  then  presented.  Based  on  the 
prior  ability  estimate  and  the  examinee's  correct  or  incorrect  response,  the  ability 
estimate  is  updated,  and  the  next  test  item  is  selected  for  presentation  from  the  new 
ability  elvel.  This  process  continues  until  the  stopping  rule  is  satisfied. 

Test  Validation 

Moreno  et  al.  (1983),  in  their  research  to  assess  the  relationship  between 
corresponding  ASVAB  and  CAT  subtests  (WK,  AR,  and  PC)  provided  a  de  facto  pilot  test 
of  CAST.  The  three  CAST  subtests  were  administered  during  a  90-day  period  late  in  1981 
to  356  male  recruits  at  the  Marine  Corps  Recruit  Depot,  San  Diego.  Each  recruit  had 
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taken  the  ASVAB  before  enlistment  and  was  retested  on  a  parallel  form  of  ASVAB  during 
recruit  processing.  After  eliminating  subjects  with  missing  scores  on  any  test  and  those 
who  had  been  administered  forms  of  ASVAB  that  were  no  longer  in  use,  the  remaining 
sample  was  270. 

In  the  CAST  pilot  test,  each  sub  test  was  administered  with  a  fixed  length:  15  items 
each  for  WK  and  AR,  and  8  items  for  PC.  All  examinees  began  with  the  same  item,  which 
was  of  medium  difficulty.  The  adaptive  tests  were  conducted  without  time  limit.  A 
preliminary  introduction  to  the  testing  situation  was  delivered  orally  by  the  proctor,  while 
all  other  instructions,  including  use  of  the  terminal  and  procedures  for  answer  entry  and 
changes,  were  presented  on  the  terminal  screen.  Practice  preceded  each  subtest  and 
successful  response  to  practice  items  was  necessary  prior  to  beginning  the  subtest.  The 
test  was  administered,  by  computer,  on  four  terminals  in  a  specially  designated  testing 
room.  The  terminals  were  on-line  with  a  Hewlett-Packard  21  MX  computer  at  the 
University  of  Minnesota,  through  a  data  communications  line. 

There  were  several  differences  between  CAST  as  it  was  designed  to  be  given  and  the 
subtest  administration  during  the  pilot  test  described  herein.  A  true  backspace  key  was 
not  available  on  the  terminal,  requiring  the  recruit  subjects  to  use  the  "Rubout"  key  to 
make  corrections.  In  the  PC  subtest,  the  stimulus  paragraphs  did  not  remain  on  the 
screen  while  the  response  alternatives  were  displayed.  The  pilot  test  was  administered  on 
terminals  communicating  with  a  host  computer,  while  CAST  is  designed  for  use  on  a 
stand-alone  microcomputer  with  attached  disk  drives.  Finally,  because  of  prior  selection 
by  the  Marine  Corps,  the  recruits  tested  were  not  representative  of  an  unselected 
applicant  population. 

After  test  scores  were  collected,  the  relationships  between  CAST  subtests  (AR,  WK, 
and  PC)  and  their  ASVAB  counterparts  were  evaluated  through  correlational  analysis.  The 
ability  of  the  CAST  subtests  to  predict  AFQT  score  was  assessed  using  multiple  regression 
analysis. 


RESULTS 

Each  CAST  sub  test  correlated  as  well  with  its  ASVAB  counterpart  as  did  the  parallel 
form  ASVAB  retest  score  (see  Table  1).  The  multiple  correlation  of  the  CAST  subtests 
with  AFQT  score  was  .866  (see  Table  2). 

Notwithstanding  the  differences  between  CAST  administration  as  originally  designed 
and  as  carried  out  in  the  Marine  recruit  testing,  the  results  of  the  recruit  testing  are 
significant.  It  was  clearly  demonstrated  that  military  recruits  (and,  by  implication, 
military  applicants)  could  be  tested  by  computer  terminal  with  minimal  intervention  by  a 
proctor.  The  CAST  subtests  measured  the  same  abilities  as  the  corresponding  ASVAB 
tests,  using  about  half  the  number  of  qustions  (Moreno  et  al.,  1983). 
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Table  1 


Tntercorrelations  Among  CAST,  Initial  ASVAB,  and  Retest  ASVAB  Tests 
of  Word  Knowledge  (WK),  Arithmetic  Reasoning  (AR), 
and  Paragraph  Comprehension  (PC)  in  a  Sample 
of  270  Marine  Recruits 


Test 

1  2  3 

4 

5 

6 

7 

8 

9 

10 

Mean 

Std.  Dev. 

1. 

Initial  ASVAB: 

AR 

1.000  0.481  0.457 

0.767 

0.417 

0.491 

0.800 

0.530 

0.429 

0.836 

21.77 

5.41 

2. 

Initial  ASVAB: 

WK 

1.000  0.573 

0.495 

0.771 

0.516 

0.501 

0.806 

0.486 

0.785 

28.17 

ON 

OO 

3. 

Initial  ASVAB: 

PC 

1.000 

0.496 

0.523 

0.464 

0.508 

0.554 

0.507 

0.687 

11.78 

2.20 

4. 

Retest  ASVAB: 

AR 

1.000 

0.483 

0.548 

0.800 

0.564 

0.495 

0.764 

21.43 

5.71 

5. 

Retest  ASVAB: 

WK 

1.000 

0.581 

0.490 

0.799 

0.533 

0.657 

28.06 

4.86 

6. 

Retest  ASVAB: 

PC 

1.000 

0.496 

0.598 

0.505 

0.618 

00 

2.50 

7. 

CAST:  AR 

1.000 

0.579 

0.525 

0.788 

0.401 

0.82 

8. 

CAST:  WK 

1.000 

0.560 

0.748 

0.588 

0.79 

9. 

CAST:  PC 

1.000 

0.542 

0.077 

0.85 

10. 

AFQT  composite 

1.000 

82.03 

11.81 

Table  2 


Correlations  of  CAST  Subtests  and  Composites  with  AFQT 

(N  =  270) 


Predictor  Variables 

Validity 

CAST:  Arithmetic  Reasoning  (AR) 

r  =  .788 

CAST:  Word  Knowledge  (WK) 

r  =  .748 

CAST:  Paragraph  Comprehension  (PC) 

r  =  .542 

Optimally  Weighted  Composite  (AR,  WK) 

R  =  .865 

Optimally  Weighted  Composite  (AR,  WK,  PC) 

R  =  .866 

DISCUSSION 

A  microcomputer-based  CAST  demonstration  system  has  been  developed,  with 
complete  documentation  for  all  software.  User-friendly,  interactive  software  provides 
full  screen  display,  clearing  the  screen  after  each  display.  Response  time  is  3  seconds  or 
less.  An  easily  used  backspace  key  and  an  error-trapping  capability  that  ensures  recruiter 
assistance  after  repeated  procedural  errors  by  the  applicant  have  been  added  to  the 
system.  Figure  3  presents  a  flow  diagram  of  the  CAST  process. 


CONGRATULATORY 

MESSAGE 


Figure  3.  Flow  chart  of  the  CAST  process. 
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Follow-on  research  and  development  with  CAST  will  evaluate  the  interactive  screen 
dialogues  for  readability  and  user  friendliness  with  service  applicants.  Capitalizing  on  the 
results  of  the  pilot  testing  of  the  subtests,  it  appears  entirely  feasibly  to  eliminate  the  PC 
subtest  from  CAST  without  a  significant  decrement  in  the  correlation  between  the  CAST 
linear  composite  and  AFQT  (see  Table  2)  (Moreno  et  al.,  1983).  Examination  of  the  data 
suggests  that  the  length  for  the  AR  and  WK  subtests  could  be  set  at  7  and  15  items 
respectively,  without  appreciable  loss  in  predictive  validity.  Were  both  procedures  to  be 
implemented,  the  correlation  between  AFQT  and  CAST  would  drop  only  to  R  =  .865  (from 
.866),  still  comparing  favorably  to  that  of  AFQT  with  the  current  EST  (r  =  .83).  The  result 
would  be  a  CAST  comprised  of  only  two  subtests,  requiring  an  average  of  16  minutes  for 
complete  administration,  scoring,  and  interpretation,  as  opposed  to  45  minutes  for  the 
EST  administration  alone,  a  significant  savings.3 


CONCLUSIONS 

The  development  of  CAST  represents  the  serendipitous  merging  of  three  elements: 
(1)  research  on  a  CAT  instrument  to  replace  the  EST,  (2)  item  banks  developed  under  a 
separate  research  effort,  and  (3)  the  planned  automation  of  recruiting  operations  that  will 
presumably  place  microcomputers  in  recruiting  stations. 

CAST  can  be  regarded  as  successfully  developed,  requiring  only  minor  refinements. 
It  should  prove  superior  to  the  EST  in  terms  of  administration  and  management  and  has 
been  shown  to  be  about  equal  in  predictive  power.  CAST  eliminates  the  need  for 
traditional  test  materials,  thereby  saving  storage  space,  replacement  costs,  and  recruiter 
time  formerly  used  for  administering  the  tests  and  controlling  and  maintaining  test 
materials.  With  CAST,  test  loss  and  theft  would  be  all  but  eliminated  and  compromise 
should  be  reduced.  Security  could  be  maintained  by  a  built-in  user  password  or 
identification.  Rather  than  serving  as  a  test  proctor-scorer,  the  recruiter  simply  would 
manage  a  computer-subject  dialogue,  with  a  self-scoring  test  for  which  results  are 
immediately  available  and  may  be  automatically  stored  for  later  use. 

Savings  in  money  and  time  would  result  by  eliminating  traditional  test  materials  and 
reducing  recruiter  time  spent  in  testing.  In  short,  CAST  will  decrease  negative 
psychological  effects,  decrease  administrative  error,  and  increase  test  security. 

CAST  is  important  not  only  for  its  present  economizing  service  but  also  for  its 
enabling  functions.  Implementing  CAST  will  allow  the  implementing  service  to  be  highly 
responsive  to  advances  in  psychometrics  and  managerial  science,  as  well  as  enable  it  to 
implement  further  applications  rapidly  when  they  are  needed.  It  is  predicted  that 
computerized  ability  testing  systems  will  find  their  optimal  use  in  organizations  serving 
populations  of  wide-ranging  ability  (De  Witt  <3c  Weiss,  1974).  With  CAST,  the  using  service 
will  be  in  the  forefront  of  CAT  implementation.  When  implemented,  CAST  will  probably 
be  the  first  large-scale  operational  use  of  computerized  adaptive  testing. 

In  today's  recruiting  climate,  where  increased  screening  capabilities  assume  ever 
greater  importance,  the  using  service  will  have  the  technological  base  upon  which  to 
mount  other  screening  instruments  for  both  selection  and  classification.  These  might 
include  (1)  predictors  of  tenure  and  effectiveness  (Sands,  1976,  1977,  1978),  (2)  assessment 
of  expectations,  intentions,  job  perceptions,  and  attitudes  (Horner,  Mobley,  &  Meglino, 
1979),  and  (3)  screening  for  specific  placement.  Screening  could  be  significantly 


3McBride,  3.  R.  Personal  communication,  October  1982. 
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improved  by  expanding  the  array  of  measures  to  include  special  abilities  and  even  biodata 
(Swanson  5c  Rimland,  1970),  since  the  administration  and  motivational  problems  associated 
with  lengthy  testing  and  examinee  fatigue  would  be  reduced  by  automation. 

Tangentially,  future  test  development  costs  and  intrusion  on  operating  systems  will 
be  reduced  because  experimental  test  items  can  be  introduced  within  CAST  in  a  manner 
that  is  transparent  to  the  field  user.  This  will  facilitate  the  development  and  evaluation 
of  new  items.  Test  administration  will  be  standardized,  fairer  to  all  applicants,  and  far 
more  efficient  in  scoring  and  recording  methods. 


RECOMMENDATIONS 

It  is  recommended  that  the  recruiting  command  of  the  using  armed  service: 

1.  Administer  CAST  to  service  applicant  populations.  Analyses  could  be  conducted 
on  the  data  obtained  to  establish  a  cutting  score  for  use  in  determining  whether  to  send  an 
applicant  to  a  military  entrance  processing  station  or  mobile  examining  test  site  for 
ASVAB  testing. 

2.  Investigate  the  suitability  of  the  CAST  interactive  dialogue,  system-user  inter¬ 
face,  and  its  effect  on  recruiting  operations. 

3.  Refine  CAST,  as  appropriate,  to  achieve  recruiter  and  applicant  acceptance. 
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